Developing a Values in Science Scale (VISS)

Background & acknowledgment

  • I’m a philosopher of science, STSer, data scientist
    • Science, values, and policy
    • Public scientific controversies
  • Emilio is a cognitive psychologist
    • Conspiracy theories & other
      “epistemically unwarranted beliefs”
    • Relationship between prejudice and EUBs
  • This project was funded out of my faculty startup funds at UC Merced

Controversy and public understanding of science

Explaining public scientific controversies

HPSTSers often explain controversies by appeal to public views on  science, values, and policy issues

But do the public hold these views?

(As a gross generalization)

  • Historians focus on actions of scientists or merchants of doubt
  • Sociologists look at very particular case studies that don’t generalize to public at large
  • Philosophers don’t offer any empirical evidence

A few notable exceptions

  • Weisberg et al. (2020)
  • Elliott et al. (2017)
    • Does transparency about values in science reduce trust?
    • Online survey experiment
    • But don’t ask about participants’ views on, eg, VFI
  • (HicksValuesDisclosuresTrust2022?)
    • Replication study of Elliott et al.
    • Found values disclosures did not reduce trust
    • Which values matter: public health vs. economic growth
    • Included a first draft of VISS, but found evidence of construct invalidity

Developing a Values in Science Scale (VISS)

VISS items

19 items, drawing on the science, values, and policy literature

Aims of science
Conflicts of interest
Consensus
Fallibilism
Inductive risk

Non-subjectivity
Pluralism
Standpoint theory
Value-free ideal

Iterative development

  • Study 1
    • Fall 2021, 980 US adults via Prolific
    • Quantitative responses to first draft of 36 items
    • Replication of Elliott et al. (2017)
    • EFA of VISS items
  • Study 2
    • March 2024
    • 5 rounds, 10 respondents per item per round
    • Qualitative responses to revised list of items
    • Two rounds focusing on conservative and HS-only respondents
  • Study 3
    • April 2024, 502 US adults via Prolific
    • Quantitative responses to revised list
    • Measures of trust in science, science literacy, right-wing authoritarianism

Agreement/disagreement

Agreement/disagreement:
Top & bottom 5

agreement item prompt
top 5
98% aims.1 A primary aim of science is to improve our knowledge and understanding of the world.
90% aims.2 A primary aim of science is to improve our understanding of threats to human health and the environment.
87% coi.2 Special interests can always find a scientist-for-hire who will support their point of view.
86% stdpt Scientists from marginalized backgrounds can provide valuable insights that other scientists are likely to miss.
79% vfi.1 The evaluation and acceptance of scientific results must not be influenced by social and ethical values.
bottom 5
35% vfi.2 Scientists do not use imagination or creativity because doing so interferes with objectivity.
35% fallible.2 When a scientific theory changes or is revised, it means that the research that went into it initially was flawed.
27% consensus.1 The consensus of the scientific community is based on social status and prestige rather than evidence.
8% fallible.1 Once a scientific theory has been established, it is never changed.
6% consensus.2 Scientists never disagree with each other about the answers to scientific questions.

Agreement/disagreement:
Interpretation

In the US, members of the public

  • think science has both epistemic and practical aims [me]
  • recognize the influence of “special interests” [Manuela, Holman]
  • agree with a version of standpoint’s epistemic advantage thesis [Crasnow]
  • nevertheless, accept VFI
  • are fallibilists [cite?]
  • recognize both epistemically-based consensus [Boaz] and legitimate scientific disagreement [Longino]

VFI and some critiques

Two statements of VFI

nonsubj.1
When analyzing data, scientists should let the data speak for itself rather than offering their own interpretation.
vfi.1
The evaluation and acceptance of scientific results must not be influenced by social and ethical values.

Four arguments against VFI

aims.2
A primary aim of science is to improve our understanding of threats to human health and the environment.
ir
Scientists should be more cautious about accepting a hypothesis when doing so could have serious social consequences.
stdpt
Scientists from marginalized backgrounds can provide valuable insights that other scientists are likely to miss.
value.conflict
When scientific findings conflict with my core values, it’s appropriate to be especially skeptical.

VFI and some critiques

VFI and some critiques: Interpretation

  • No correlation between statements of VFI and critiques
  • Not:
    Public is irrational, inconsistent, etc.
  • Rather:
    Public haven’t yet had the opportunity to think through these issues

Future work

Future work

  • Qualitative designs to revise prompts
    • Paraphrase the prompt; think aloud
    • Likely will reduce measurement error
  • Other endpoints
    • Dueling expert scenarios
    • Trust in science-policy institutions (EPA) rather than individual scientists (Brown 2022; Biddle yesterday)
    • Policy support

Fin

values-in-science-scale.netlify.app

Extra slides

Prompts

aims-1
The knowledge produced by scientific research is valuable even when it has no practical applications.
aims-2
A primary aim of science is to improve our understanding of threats to human health and the environment.
aims-3
A primary aim of science is to develop new technology and stimulate economic growth.
coi-1
Scientists will report conclusions that they think will get them more grant money even if the data does not fully support that conclusion.
coi-2
Special interests can always find an scientist-for-hire who will support their point of view.
coi-3
Research carried out by private companies is less trustworthy than research carried out by public instutitions.
consensus-1
Progress in science happens when a lone dissenter challenges popular ideas.K
consensus-2
Scientists rarely disagree with each other about the answers to scientific questions.
consensus-3
The consensus of the scientific community is based on social status and prestige rather than evidence.
factvalue-1
Moral and social values are the kinds of things that can be confirmed or challenged by scientific evidence.
factvalue-2
If scientific findings conflict with important values that we hold, it’s appropriate to be skeptical of the science.
factvalue-3
Someone’s moral and social values are more like their taste in music than scientific facts.
fallible-1
Once a scientific theory has been established, it is never changed.K
fallible-2
When a scientific theory changes or is revised, it means that the research that went into it initially was flawed.
fallible-3
Scientific findings can be good enough to act on even when they’re not entirely certain.
ir-1
Standards of scientific evidence are different in different situations.
ir-2
Scientists are not responsible for harmful effects of the technologies they develop.
ir-3
When scientific research has potential consequences for society, scientists must take these consequences into account when drawing conclusions.
nonsubj-1
The results of scientific studies are always ambiguous and require interpretation.
nonsubj-2
Good scientific research is always free of assumptions and speculation.
nonsubj-3
Scientific conclusions are never a matter of interpretation or judgment.
pluralism-1
Scientific investigations always require laboratory experiments.K
pluralism-2
Scientific research is always conducted in the following order: 1. Observation, 2. Hypothesis, 3. Experiment, 4. ConclusionK
pluralism-3
All scientists use the same strict requirements for determining when empirical data confirm a tested hypothesis.
scientism-1
Science is more reliable than any other way of producing knowledge.
scientism-2
Everyone, whatever their worldview, should accept evidence-based scientific conclusions
scientism-3
Science is the only source of justified belief or knowledge about ourselves and the world.
stdpt-1
A scientific understanding of the social word depends on the background of the scientists carrying out the research.
stdpt-2
People on the margins of society often have a unique understanding of that society that can benefit scientific research.
stdpt-3
Science has a long history of ignoring the unique insights of women and people of color.
technocracy-1
Public policy should wait until all relevant scientific questions have been settled.
technocracy-2
When it comes to public policy, scientific evidence is more important than public opinion.
technocrac-3
Scientists have too much influence over public policy today.
vfi-1
Scientists are more objective than other people.K
vfi-2
Scientists do not use imagination or creativity because doing so interferes with objectivity.K
vfi-3
The evaluation and acceptance of scientific results must not be influenced by social and ethical values.K

Factor analysis results

VISS factors

Screenshot of the factor correlation matrix. The factors are labeled scientism, VIS (values in science), cynicism, power, textbook (science), and VFI (value-free ideal)

⚠️ Psychometric properties of these factors aren’t great ⚠️

Factor analysis fit indices

How well does the fitted FA model approximate the observed covariance structure? (Hu and Bentler 1998)


statistic range want our model
Comparative fit index (CFI) 0-1 large 0.6
Adjusted-goodness-of-fit index (AGFI) 0-1 large 0.8
Root-mean-square error of approximation (RMSEA) >0 small 0.07
Standardized root-mean-square residual (SRMSR) >0 small 0.09

Scientism

Density plot for scientism factor scores. The mode is just below 3.5, and values range from 1 to 5.

  • Science is more reliable than any other way of producing knowledge.
  • Scientific findings can be good enough to act on even when they’re not entirely certain.
  • The knowledge produced by scientific research is valuable even when it has no practical applications.

Plot of scientism against political ideology. There's a weak negative association: on average conservatives are about 0.5 point less scientistic than liberals.

  • Scientists are not responsible for harmful effects of the technologies they develop.
  • When it comes to public policy, scientific evidence is more important than public opinion.
  • Moral and social values are the kinds of things that can be confirmed or challenged by scientific evidence.

Cynicism

Density plot for cynicism factor scores. The mode is about 2.5, and values range from 1 to about 4.5.

  • Scientists will report conclusions that they think will get them more grant money even if the data does not fully support that conclusion.K
  • The consensus of the scientific community is based on social status and prestige rather than evidence.
  • If scientific findings conflict with important values that we hold, it’s appropriate to be skeptical of the science.
  • The results of scientific studies are always ambiguous and require interpretation.

Plot of cynicism against political ideology. There's a positive association: on average conservatives are about 1 point more cynical than liberals.

  • When a scientific theory changes or is revised, it means that the research that went into it initially was flawed.
  • Special interests can always find an scientist-for-hire who will support their point of view.
  • Standards of scientific evidence are different in different situations.
  • A scientific understanding of the social world depends on the background of the scientists carrying out the research.

Factors and trust

Estimated effects plot for all six VISS factors. Power, VFI, and VIS factors are essentially flat. Scientism and textbook are associated with increased trust, while cynicism is associated with decreased trust. However, all effects are modest, highly unlikely to be greater than +/- 1 on the 1-7 trust scale.

References

Brown, Matthew. 2022. “Trust, Expertise and Scientific Authority in Democracy.” Michigan State University, February 20. https://www.youtube.com/watch?v=y3XeP6e646g.
Cook, John, Dana Nuccitelli, Sarah A. Green, Mark Richardson, Bärbel Winkler, Rob Painting, Robert Way, Peter Jacobs, and Andrew Skuce. 2013. “Quantifying the Consensus on Anthropogenic Global Warming in the Scientific Literature.” Environmental Research Letters 8 (2): 024024. https://doi.org/10.1088/1748-9326/8/2/024024.
Elliott, Kevin C., Aaron M. McCright, Summer Allen, and Thomas Dietz. 2017. “Values in Environmental Research: Citizens’ Views of Scientists Who Acknowledge Values.” PLOS ONE 12 (10): e0186049. https://doi.org/10.1371/journal.pone.0186049.
Elliott, Kevin C., and David B. Resnik. 2014. “Science, Policy, and the Transparency of Values.” Environmental Health Perspectives, March. https://doi.org/10.1289/ehp.1408107.
Fernandez Pinto, Manuela. 2020. “Commercial Interests and the Erosion of Trust in Science.” Philosophy of Science, July. https://doi.org/10.1086/710521.
Fernández Pinto, Manuela, and Daniel J. Hicks. 2019. “Legitimizing Values in Regulatory Science.” Environmental Health Perspectives 127 (3): 035001. https://doi.org/10.1289/EHP3317.
Goldenberg, Maya J. 2021. Vaccine Hesitancy: Public Trust, Expertise, and the War on Science. University of Pittsburgh Press.
Grasswick, Heidi. 2018. “Understanding Epistemic Trust Injustices and Their Harms.” Royal Institute of Philosophy Supplements 84 (November): 69–91. https://doi.org/10.1017/S1358246118000553.
Hu, Li-tze, and Peter M. Bentler. 1998. “Fit Indices in Covariance Structure Modeling: Sensitivity to Underparameterized Model Misspecification.” Psychological Methods 3 (4): 424–53. https://doi.org/10.1037/1082-989X.3.4.424.
John, Stephen. 2017. “Epistemic Trust and the Ethics of Science Communication: Against Transparency, Openness, Sincerity and Honesty.” Social Epistemology 32 (2): 75–87. https://doi.org/10.1080/02691728.2017.1410864.
Jordan, Catherine, Susan Gust, and Naomi Scheman. 2011. “The Trustworthiness of Research: The Paradigm of Community-Based Research.” In Shifting Ground: Knowledge and Reality, Transgression and Trustworthiness, by Naomi Scheman, 170–90. Oxford University Press. http://books.google.ca/books?id=ZvqLRu30ipsC&pg=PA149&dq=intitle:Shifting+Ground+inauthor:scheman&hl=&cd=1&source=gbs_api.
Kovaka, Karen. 2021. “Climate Change Denial and Beliefs about Science.” Synthese 198 (3): 2355–74. https://doi.org/10.1007/s11229-019-02210-z.
Largent, Mark A. 2012. Vaccine: The Debate in Modern America. JHU Press.
Lombrozo, Tania, Anastasia Thanukos, and Michael Weisberg. 2008. “The Importance of Understanding the Nature of Science for Accepting Evolution.” Evolution: Education and Outreach 1 (3): 290–98. https://doi.org/10.1007/s12052-008-0061-8.
McHugh, Nancy Arden. 2011. “More Than Skin Deep: Situated Communities and Agent Orange in the Aluoi Valley, Vietnam.” In Feminist Epistemology and Philosophy of Science, edited by Heidi E Grasswick, 183–204. Power In Knowledge. Springer. http://books.google.com/books?id=fsLVt65PxrAC&printsec=frontcover&dq=intitle:Feminist+Epistemology+and+Philosophy+of+Science&hl=&cd=1&source=gbs_api.
Mercer, David. 2016. “Why Popper Can’t Resolve the Debate over Global Warming: Problems with the Uses of Philosophy of Science in the Media and Public Framing of the Science of Global Warming.” Public Understanding of Science, May, 0963662516645040. https://doi.org/10.1177/0963662516645040.
Myers, Krista F., Peter T. Doran, John Cook, John E. Kotcher, and Teresa A. Myers. 2021. “Consensus Revisited: Quantifying Scientific Agreement on Climate Change and Climate Expertise Among Earth Scientists 10 Years Later.” Environmental Research Letters 16 (10): 104030. https://doi.org/10.1088/1748-9326/ac2774.
Navin, Mark. 2015. Values and Vaccine Refusal: Hard Questions in Ethics, Epistemology, and Health Care. Routledge.
Nelson, Alondra. 2011. Body and Soul: The Black Panther Party and the Fight Against Medical Discrimination. U of Minnesota Press.
Oreskes, N. 2004. “The Scientific Consensus on Climate Change.” Science 306 (5702): 1686. https://doi.org/10.1126/science.1103618.
Scheman, Naomi. 2001. “Epistemology Resuscitated: Objectivity as Trustworthiness.” Engendering Rationalities, 23–52. http://books.google.com/books?hl=en&lr=&id=CLl5HMhRWBsC&oi=fnd&pg=PA23&dq=epistemology+resuscitated+naomi+scheman&ots=2RBRcsu4dg&sig=h7xRsp8flRZFpsFW3rCfCw9R7C4.
Washington, Harriet A. 2008. Medical Apartheid: The Dark History of Medical Experimentation on Black Americans from Colonial Times to the Present. Knopf Doubleday Publishing Group.
Weisberg, Deena Skolnick, Asheley R. Landrum, Jesse Hamilton, and Michael Weisberg. 2020. “Knowledge about the Nature of Science Increases Public Acceptance of Science Regardless of Identity Factors.” Public Understanding of Science, December, 0963662520977700. https://doi.org/10.1177/0963662520977700.
Wynne, Brian. 1989. “Sheepfarming After Chernobyl: A Case Study in Communicating Scientific Information.” Environment: Science and Policy for Sustainable Development 31 (2): 10–39. https://doi.org/10.1080/00139157.1989.9928930.